Skip to content

[New Package] Add FreeIPA integration#17812

Closed
Oddly wants to merge 1 commit into
elastic:mainfrom
Oddly:add-freeipa
Closed

[New Package] Add FreeIPA integration#17812
Oddly wants to merge 1 commit into
elastic:mainfrom
Oddly:add-freeipa

Conversation

@Oddly

@Oddly Oddly commented Mar 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Initial release of the FreeIPA (Red Hat IdM) integration. Collects security-relevant logs from five FreeIPA subsystems using filestream input: Kerberos KDC authentication events, 389 Directory Server access and error logs, Dogtag CA signed audit events, and IPA JSON API operations. Includes five ES|QL alerting rule templates that activate on Stack 9.2+, three optional transforms for cross-event LDAP correlation, and comprehensive ECS field mappings including x509 certificate fields and rule.name for policy objects. Dashboards will follow in a separate PR.

Changes

  • Five data streams: kdc, directory_access, directory_errors, ca_audit, ipa_api
  • Ingest pipelines with full ECS mapping, GeoIP enrichment on source IPs, and tagged processors throughout
  • KDC pipeline handles AS_REQ/TGS_REQ with constrained delegation (S4U2Proxy) support, maps to source.user/user.target for delegation events
  • Directory access pipeline parses all LDAP operation types (BIND, SRCH, MOD, ADD, DEL, CMP, EXT, MODRDN, ABANDON, conn=Internal) with wtime/optime/etime extraction and unindexed search detection
  • CA audit pipeline parses Dogtag signed audit log with certificate lifecycle tracking, CRL metrics (crl_number/crl_size), and ECS x509 field mapping (file.x509.serial_number, subject/issuer distinguished_name)
  • IPA API pipeline extracts target entities and maps to ECS fields per command family: user.target.name, group.name, host.name, rule.name (sudorule/hbacrule), related.hosts (DNS/service), with per-family event.category refinement
  • Five alerting rule templates: replication errors, unindexed LDAP searches, LDAP high latency, API error spike, CA connection failures
  • Three optional transforms (latest_bind, latest_connection, latest_auth) with lookup-mode destination indices for ES|QL LOOKUP JOIN enrichment in detection rules
  • 100% pipeline branch coverage across all data streams, zero dead paths

Testing

  • Deployed on two FreeIPA 4.11 servers (Rocky Linux 9, primary + replica) with Elastic Agent 8.19 enrolled via Fleet
  • All five data streams confirmed receiving live data (589 KDC events, 11k directory access, 1.6k directory errors, 564 CA audit, 73 IPA API)
  • All defined fields populated with correct types in indexed data — zero type mismatches, zero missing fields
  • All five alerting rule ES|QL queries executed against live data and returned correct results:
    • Replication errors: triggered with synthetic replication bind failures
    • Unindexed searches: triggered by searching on non-indexed attributes (roomNumber, carLicense)
    • LDAP high latency: query valid, correctly silent on healthy system (p95 = 7ms vs 5s threshold)
    • API error spike: triggered with user_show calls for nonexistent users
    • CA connection failures: caught 48 real ConnectException events from CA startup
  • 24 of 27 detection rules confirmed firing against live data, 0 false positives
  • Pipeline tests pass 18/18 for all data streams covering all operation types and edge cases
  • elastic-package check passes (lint + build), full_review passes with 0 errors/0 warnings

@Oddly Oddly requested a review from a team as a code owner March 14, 2026 09:36
@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Mar 14, 2026

Copy link
Copy Markdown
Contributor

Reviewers

Buildkite won't run for external contributors automatically; you need to add a comment:

  • /test : will kick off a build in Buildkite.

NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details.

Oddly added a commit to Oddly/integrations that referenced this pull request Mar 14, 2026
@Oddly
Update changelog PR link to elastic#17812
36e8511
@andrewkroh andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. New Integration Issue or pull request for creating a new integration package. labels Mar 15, 2026
@Oddly Oddly mentioned this pull request Mar 28, 2026
Draft
@Oddly Oddly force-pushed the add-freeipa branch 2 times, most recently from 3b030cf to e5ab704 Compare March 29, 2026 20:06
macroscopeapp[bot]
macroscopeapp Bot reviewed Apr 1, 2026
View reviewed changes
Comment thread packages/freeipa/data_stream/kdc/sample_event.json
macroscopeapp[bot]
macroscopeapp Bot reviewed Apr 1, 2026
View reviewed changes
Comment thread packages/freeipa/elasticsearch/transform/latest_bind/transform.yml
macroscopeapp[bot]
macroscopeapp Bot reviewed Apr 1, 2026
View reviewed changes
Comment thread packages/freeipa/data_stream/directory_errors/elasticsearch/ingest_pipeline/default.yml Outdated
@Oddly
[New Package] Add FreeIPA integration
69950cd 
Adds a comprehensive FreeIPA integration with five log data streams
covering all major FreeIPA services:

  kdc — Kerberos KDC authentication events (TGS/TGT grants, failures,
        S4U proxy, pre-auth, encryption types)
  directory_access — 389 Directory Server access log (BIND, SEARCH,
        MOD, ADD, DELETE, MODRDN with connection tracking)
  directory_errors — 389 DS error log (replication failures, plugin
        errors, resource warnings)
  ca_audit — Dogtag CA signed audit log (certificate requests,
        approvals, revocations, CRL generation, ACL decisions)
  ipa_api — IPA JSON/XML-RPC API access log (commands, parameters,
        results, caller identity)

Includes two transforms for connection-level LDAP analysis (latest
bind state and operation enrichment), GeoIP enrichment on client IPs,
ECS-compliant field mappings, and pipeline tests with both synthetic
and real-world log samples from a multi-replica FreeIPA deployment.
macroscopeapp[bot]
macroscopeapp Bot reviewed Apr 1, 2026
View reviewed changes
Comment thread packages/freeipa/elasticsearch/transform/latest_auth/fields/ecs.yml
@andrewkroh andrewkroh removed the New Integration Issue or pull request for creating a new integration package. label Apr 23, 2026
@andrewkroh andrewkroh added the New Integration Issue or pull request for creating a new integration package. label May 11, 2026
@botelastic

botelastic Bot commented Jun 10, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic Bot added the Stalled label Jun 10, 2026
@botelastic

botelastic Bot commented Jul 10, 2026

Copy link
Copy Markdown

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic Bot closed this Jul 10, 2026
@Oddly

Oddly commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

It would surely be nice to finally have FreeIPA in our SIEM. Why isn't this a priority?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@macroscopeapp macroscopeapp[bot] macroscopeapp[bot] left review comments

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. New Integration Issue or pull request for creating a new integration package. Stalled

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Oddly @andrewkroh